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I.   INTRODUCTION 

Trident  CCSMA  has  requested  the  Naval  Postgraduate  School  to  evaluate 
the  Department  of  the  Navy's  Tactical  Digital  Systems  Documentation 
Standard  SEGNAVINST  3560.1  dated  8  August  1974,  hereafter  referred  to  as 
3560.1,  with  respect  to  its  applicability  and  usefulness  for  software 
maintenance.   This  report  is  divided  into  the  following  sections: 

Brief  Description  of  3560.1.   This  section  is  provided  to  give  the 
reader  who  is  unfamiliar  with  3560.1  a  brief  overview  of  its 
contents. 

-  Traceability .   Since  a  major  concern  of  CCSMA  is  traceability , 
3560.1  is  evaluated  with  respect  to  its  traceability  attributes. 
Traceability  exists  when  it  is  possible  to  identify  the  parts  of 
the  software  system,  and  the  corresponding  documentation,  that 
will  be  affected  by  a  change  in  the  software  stemming  from  a 
software  error  correction  or  enhancement . 

-  Usefulness  of  3560.1  for  Supporting  Software  Maintenance.  Each 
section  of  3560.1  that  is  applicable  to  software  maintenance  is 
examined  with  respect  to  usefulness  for  maintenance. 

-  Conclusions  and  Recommendations .   Major  conclusions  concerning 
the  adequacy  of  3560.1  for  software  maintenance  are  drawn  and 
recommendations  are  made  to  make  it  more  suitable  for  this 
activity. 

The  major  conclusion  of  this  report  is  that,  as  it  stands, 
3560.1  is  inadequate  for  effectively  supporting  a  software 


maintenance  activity.   However,  with  the  improvements  that  have 
been  recommended,  3560.1  could  be  the  equal  of  recently  announced 

tactical  software  standards. 


II.   BRIEF  DESCRIPTION  OF  3560.1 

1.  Tactical  Operational  Requirement  (TOR) 

Tactical  digital  system  functional  requirements . 

Serves  as  a  basic  software  specification  for  design  and  program 

implementation . 

2.  System  Operational  Specification  (SOS) 

Specific  operations  desired  from  application  of  the  digital 
processor  supported  data  system. 

3.  System  Operational  Design  (SOD) 

Plan  for  integrated  system  program. 

Program  functions ,  core  allocations ,  subprogram  definitions  and 

interfaces,  program  data  storage  plans,  and  support  programs. 

-  I/O  channel  assignments. 

Data  to  be  exchanged  between  digital  processors  and  peripherals. 

-  Timing  requirements  for  messages. 

4.  Function  Operational  Specification  (FOS) 

Design  of  each  separate  data  function. 

Specify  each  required  action  at  each  operator's  position. 

5 .  Interface  Design  Specification  (IDS) 

-  Specifications  for  interdigital  processor  message  traffic  in 
format  and  content. 

6.  Program  Performance  Specification  (PPS) 

-  Describes  performance  requirements  for  the  computer  programs 
of  the  digital  processor  system. 


Baseline  for  configuration  control. 
Controlling  document  for  procuring  agency. 

7 .  Function  Operational  Design  (FOP) 

Program  performance  design  in  operator  function  terms  for 
each  operator  and  peripheral  equipment. 

8.  Program  Design  Specification  (PDS) 

Design  details  for  digital  processor  programs  in  programming 

language . 

Used  for  program  maintenance . 

9.  Program  Description  Document  (PDD) 

Design  details  for  each  subprogram. 

10.  Data  Base  Design  (DBD) 

Detailed  description  of  all  data  items. 

11.  Program  Package  (PP) 

Card  decks,  tapes,  listings,  etc. 

12.  Test  Plan  (TPL) 

Defines  the  scope  of  tests  required  to  ensure  that  the  system, 
function  and  program  meet  all  applicable  technical,  operational, 
and  performance  specifications. 
-  Establishes  detailed  acceptance  criteria   for  the  system  and 
identifies  each  level  of  testing. 

13.  Test  Specification  (TS) 

Purpose  and  scope  of  test. 

Identifies  software,  hardware,  and  system  to  be  tested. 

14.  Test  Procedure  (TPP) 

Instructions  for  test  execution  and  evaluation  of  results. 


15.  Test  Report  (TR) 

Describes  and  evaluates  discrepancies  between  program  design  and 
operation. 

16.  Operator's  Manual  (OM) 

Presents  procedures  for  prestandby,  operate,  monitoring,  and 
recovery  of  the  digital  processor  program. 
Describes  the  minimal  operating  environment. 

17.  Program  Design  Manual  (PPM) 

Provides  the  theory  of  combat  direction  system  program  processes 
that  support  the  station  operator. 

By  operator  and  equipment  function ,  describes  the  digital 
processor  program  logic  and  algorithms  that  produce  actions  and 
data  displays  for  the  operator. 

18.  Command  and  Staff  Manual 

Provides  a  nontechnical  description  of  the  tactical  system. 
Addresses  the  mission,  characteristics,   employment,  capabilities 
and  limitations  of  the  tactical  system. 

19.  System  Operator's  Manual 

-  Sole  reference  for  individual  operator  duties  and  station 
function. 

Explains,  at  the  level  required  by  system  operators,  every 
control  button,  switch,  readout,  and  display  affected  by  the 
system  program. 


III.   TRACE ABILITY 

In  order  to  illustrate  the  traceability  characteristics  of  3560.1/ 
TABLE  1  is  provided  to  show  the  specific  references  which  a  given  section 
of  3560.1  (e.g.,  SOS)  makes  to  the  other  section  (s)  (e.g.,  TOR).   Where 
these  references  occur,  an  "X"  is  placed  in  the  appropriate  cell  of  TABLE 
1.   The  table  is  read  by  interpreting  the  left-hand  column  as  the  section 
in  which  a  reference  occurs,  and  the  top  row,  where  there  is  an  "X"  in 
a  cell,  to  be  a  referenced  section.   The  resultant  matrix  indicates  the 
degree  of  traceability  that  exists  in  the  standard.   That  is,  the  density 
or  sparseness  of  the  matrix  is  one  measure  of  the  existence  or  absence  of 
traceability,  respectively. 

The  desired  traceability  relationship,  as  implied  ty  "TDS  Documenta- 
tion Relationship,"  Figure  2  on  page  6  of  3560.1,  is  shown  in  FIGURE  1. 
The  chart  (FIGURE  1)  was  derived  from  Figure  2  of  3560.1  by  considering 
only  those  sections  of  the  standard  that  are  concerned  with  program 
development,  design,  and  testing,   The  arrows  are  upward  pointing  to 
suggest  the  use  of  documentation  for  traceability  purposes.   For  example, 
in  FIGURE  1,  a  Test  Report  (TR)  can  be  traced  to  the  Tactical  Operational 
Requirement  (TOR).  The  actual  traceability  relationships,  as  defined  by 
TABLE   1  for  the  documents  shown  in  FIGURE  1,  are  shown  in  FIGURE  2. 
Although  a  great  deal  of  traceability  capability  exists  in  3560.1,  a 
comparison  of  FIGURE  1  with  FIGURE  2  shows  that  actual  traceability  is 
less  than  desired  traceability,  thus  indicating  inconsistencies  between 
the  objectives  of  the  standard,  relative  to  traceability,  and  the  actual 

content  of  its  various  sections . 
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Although  it  is  not  the  conventional  approach,  the  argument  can  be 
made  that  not  even  the  "desired  traceability  relationship"  shown  in 
FIGURE  1  is  complete  because  traceability,  as  it  is  shown,  depends  on  a 
long  chain  of  related  documents.   For  example,  the  Test  Procedure  (TPR) 
should  be  related  back  to  the  Tactical  Operational  Requirement  (TOR) , 
which  is  written  in  operational  user  language,  if  the  Test  Procedure  is 
to  be  a  true  reflection  of  user  requirements.   In  other  words,  it  should 
be  easy  to  see  how  the  Test  Procedure  meets  user  requirements ,  rather 
than  trace  through  several  intervening  documents  in  order  to  discern 
this  relationship.   In  addition,  the  intervening  document (s)  could  con- 
tain errors,  which  would  cause  a  distorted  view  of  how  the  TPR  is  to 
meet  user  requirements. 
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FIGURE  1.     DESIRED  TRACEABILITY  RELATIONSHIP. 
*  MISSING  IN  FIGURE  2. 
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FIGURE  2,  ACTUAL  TRACEABILITY  RELATIONSHIP  BASED  ON 
REFERENCES  AMONG  SECTIONS, 
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IV.   USEFULNESS  OF  3560.1  FOR  SUPPORTING  SOFTWARE  MAINTENANCE 

Traceability ,  which  was  covered  in  Section  III,  is  a  capability 
which  is  useful  for  both  software  development  and  maintenance.   In  this 
section,  a  sepcific  examination  is  made  of  the  adequacy  of  3560.1  for 
software  maintenance.   Each  section  of  3560.1  that  either  mentions  main- 
tenance or  could  be  useful  for  software  maintenance  is  analyzed  below. 
Recommendations  for  improvements  in  the  coverage  of  software  maintenance 
are  noted.   Section  and  page  references  are  to  3560.1. 

TABLE  2  is  provided  to  summarize  those  sections  of  3560.1  that  are 
applicable  to  software  maintenance.   The  table  shows  section,  section 
number  and  page  references,  purpose  of  the  section,  and  an  indication  of 
whether  the  section  is  adequate  for  software  maintenance.   Brief  remarks 
are  given,  where  appropriate.   In  the  case  of  negative  remarks,  expla- 
nations are  provided  following  TABLE  2 . 

1.  Tactical  Operational  Requirement  Section  1.5.4  Test  Programs, 
p.   1-13. 

Reference  is  made  to  any  maintenance  programs  that  might  be 

required.   The  paragraph  seems  to  refer  to  software  that  is  used  to 

maintain  hardware.   Specific  reference  should  be  made  to  the  need  to 

provide  software  tools  for  maintaining  software  in  addition  to  the 

software  used  for  maintaining  hardware. 

2.  Interface  Design  Specification  Section  1.0  Purpose,   p.  2-41. 
This  section  states:   "Upon  completion  of  program  development, 

the  Interface  Design  Specification  shall  serve  as  a  joint  life  cycle 
configuration  control  device  for  digital  processor  program  maintenance 
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of  the  interface."   Except  for  the  section  noted  in  TABLE  2  and  the 
corresponding  explanation  of  remarks,  the  IDS  contains  good  sections 
for  supporting  software  maintenance.   This  is  due  primarily  to  the 
thoroughness  of  treatment  of  items,  such  as  signal  definitions  and  inter - 
processor  communications. 

3.  Program  Performance  Specification  Section  1.0  Purpose,   p.  2-33. 
This  section  states:   "The  Program  Performance  Specification  shall 

describe  in  detail  all  the  operational  and  functional  requirements 
necessary  to  design,  test,  and  maintain  the  required  digital  processor 
program."  As  shown  in  TABLE  2  and  the  explanation  of  remarks,  there  are 
sections  of  the  PPS  which  are  redundant,  unclear,  or  require  more  empha- 
sis on  validation  as  opposed  to  verification.   With  respect  to  the  last 
item,  see  the  following  section  on  the  Program  Description  Document, 
which  contrasts  validation  with  verification.   The  comments  in  that 
section  apply  as  well  to  the  PPS.   For  these  reasons  the  PPS  is  not 
entirely  adequate  for  software  maintenance . 

4.  Program  Description  Document  Section  1.0  Purpose,   p.  2-137. 
This  section  states:   "As  a  detailed  compendium  of  the  subprogram 

structure,  the  Program  Description  document  will  serve  as  the  essential 
instrument  for  subsequent  use  by  operational ,  maintenance,  and  contractor 
personnel  diagnosing  troubles,  making  adaption  changes,  designing  and 
implementing  modifications  to  the  system,  and  in  introducing  or  adding 
new  subprogram  functions  to  the  completed  program."   Thus,  the  PDD  is 
one  of  the  major  documents  which  govern  the  conduct  of  software  mainte- 
nance. As  shown  in  TABLE  2  and  in  the  corresponding  explanation  of  re- 
marks, the  quality  assurance  provisions  of  the  PDD  provide  sufficient 
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emphasis  on  verifying  programs,  via  testing,  but  insufficient  emphasis 
on  validating  programs  against  tactical  requirements  (e.g.,  TOR).   As 
defined  by  Lewis:   "Verification  is  the  iterative  process  aimed  at 
determining  whether  the  product  of  each  step  in  the  software  develop- 
ment cycle  fulfills  all  the  requirements  levied  by  the  previous  step: 
Does  B  fulfill  requirements  of  A?   Does  C  fulfill  requirements  of  B,  and, 
implicitly,  fulfill  requirements  of  A?  ...  .   Validation  is  essentially 
that  part  of  verification  and  validation  which  looks  back  at  the  software 
requirements  and  determines  through  testing  that  they  are  (or  are  not) 
satisfied  by  the  observable  system  performance  indicators.   The  impli- 
cation of  validation  is  that  the  system  will  meet  its  operational  life 
cycle  design  commitments."    Thus  the  PDD  is  not  entirely  adequate  for 
software  maintenance,  although  it  does  contain  many  comprehensive  and 
detailed  sections. 

5.   Program  Package  Section  1.0  Purpose,  p.  2-165. 

This  section  states:   "The  Program  Package  document  shall  con- 
sist of  all  the  program  material  items  necessary  for  the  procuring  agency 
to  produce,  maintain,  and  update  the  digital  processor  program."   Several 
sections  of  the  PP  refer  to  card  decks  and  magnetic  tapes  as  the  media  for 
source  programs.   These  sections  should  be  augmented  to  allow  for  the 
possibility  of  other  physical  media,  such  as  disc  pack,  cassette, 
cartridge,  and  ROM.   In  addition,  the  possible  use  of  interactive  compila- 
tion and  debugging  facilities  for  software  production  and  the  storing 


Robert  0.  Lewis,  "Software  Verification  and  Validation,"  in 
John  D.  Cooper  and  Mathew  Fisher  (Editors) ,  Software  Quality  Management, 
Chapter  15 . 
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of  program  files  in  a  time -sharing  facility  should  be  recognized.   Finally, 
the  document  should  allow  for  the  possibility  of  computer  to  computer 
transfer  of  program  files,  without  the  intervening  physical  media  of 
cards  and  tapes . 

6.   Operator's  Manual  Section  1.0  Purpose,  p.  4-5. 

This  section  states:   "The  Operator's  Manual  shall  be  limited  to 
instructions  for  preparing  and  maintaining  the  digital  processor  program 
in  the  required  state  of  capability  in  order  that  the  operational 
mission  may  be  accomplished."   As  shown  in  TABLE  2,  there  are  no  sections 
of  the  OM  that  are  considered  inadequate  for  software  maintenance. 
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TABLE  2 

SUMMARY  OF  3  560.1  SECTIONS 
APPLICABLE  TO  SOFTWARE  MAINTENANCE 

INPUTS 


SEC 


SEC  # 


PAGE 


PURPOSE 


eY/N 


REMARKS 


FOS 

3.3.1 

2-37 

CHARACTERISTICS 

N 

UNCLEAR 

PPS 

3.4.N.1 

2-95 

CHARACTERISTICS 

Y 

PDD 

3.4 

2-146 

FORMATS 

Y 

SOM 

N 

4-38 

DATA  ENTRY 

Y 

PROCESSING 


FOS 

3.3.2 

2-37 

PROGRAM  PROCESS 

PPS 

3.4.N.2 

2-98 

FUNCTION  PROCES, 
OUTPUTS 

FOS 

3.3.3 

2-37 

DISTRIBUTION 

PPS 

3.4.N.3 

2-99 

CHARACTERISTICS 

PDD 

3.4 

2-146 

FORMATS 

TS 

3.2.4 

3-29 

TEST  OUTPUTS 
FUNCTIONS 

PPS  3.3  2-90  FUNCTION  DESCRIPTION 

PPS  3.3.5  2-92  FUNCTION  DESCRIPTION 

PPS  3.4  2-95  FUNCTION  REQUIREMENT 

PPS  TABLE  3-11  2-97  FUNCTION  VS.  STATE 

PDS  3.1  2-123  FUNCTION  REQUIREMENT 

PDS  3.2  2-124  FUNCTION  DESCRIPTION 

PDS  3.4  2-127  FUNCTION  FLOW 


N 

REDUNDANT 

N 

REDUNDANT 

N 

REDUNDANT 

Y 

GOOD  TABLE 

Y 

Y 

Y 

*Y/N  COLUMN: 


"Y"  MEANS  SECTION  IS  ADEQUATE  FOR  SOFTWARE  MAINTENANCE. 
"N"  MEANS  SECTION  IS  INADEQUATE  FOR  SOFTWARE  MAINTENANCE. 
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DATABASE 


SEC 


SEC  # 


PAGE 


PPS 

3.5 

2-99 

PDD 

3.3.2 

2-145 

DBD 

ALL 

2-153 

PURPOSE 


Y/N 


REQUIREMENTS  N 

SUBPROGRAM  DATA  DESIGN    N 
COMMON  DATA  DESCRIPTION   Y 


REMARKS 

UNCLEAR 
TERMINOLOGY 


interface: 


SOS 

5.2 

2-14 

SOS 

5.3 

2-15 

SOS 

5.4 

2-15 

SOD 

5.2 

2-28 

SOD 

5.3 

2-28 

SOD 

5.4 

2-28 

IDS 

1.0 

2-41 

IDS 

2.0- 

•8.3 

2-42 

PPS 

3.2. 

3 

2-89 

PPS 

3.3. 

3 

2-92 

PPS 

3.3. 

4 

2-92 

PDD 

3.8 

2-149 

PDD 

FIG. 

3-2 

2-150 

PERIPHERAL   SYSTEM  N 

OPERATOR  N 

INTERS Y STEM  Y 

PERIPHERAL   SYSTEM  Y 

OPERATOR  N 

INTERSYSTEM   ON-LINE  Y 

INTERFACE    REQUIREMENTS  N 

INTERFACE    REQUIREMENTS  Y 

INTERFACE    I.D.  Y 

DIG.    PROC.    BLOCK  DIA.  Y 

PROGRAM  Y 

SUBPROGRAMS  Y 

SUBPROGRAMS  Y 


VAGUE 
VAGUE 


INCOMPLETE 

CONFUSING 
GOOD  SECTIONS 


TESTING 


PPS 

3.4.N.4 

PPS 

4.2 

FOD 

5 

TPL 

1.0 

TPL 

1.1.2 

TPL 

1.1.3 

TS 

3.2 

TPR 

1.0 

TPR 

3.2.1 

TPR 

3.2.3 

TPR 

7 

TR 

ALL 

2-99 

2-101 
2-115 
3-5 

3-7 

3-8 

3-23 

3-35 

3-40 

3-41 

3-43 

3-45 


SPECIAL  REQUIREMENTS 

Y 

TEST  REQUIREMENTS 

Y 

TEST  &  SIM.  SCENARIOS 

Y 

SCOPE  &  LEVELS 

N 

EQUIPMENT 

Y 

SUPPORT  SOFTWARE 

Y 

SYS. /PROG.  DEFINITION 

Y 

INSTRUCTIONS  &  EVAL. 

N 

EQUIPMENT  PREPARATION 

Y 

TEST  PROCEDURE 

N 

SUPPORT  SOFT.  REQUIRE. 

Y 

TEST  REPORTS 

Y 

TERMINOLOGY 
VALIDATION  NEEDED 


UNCLEAR 


UNCLEAR 


EXCEPT 

PATCH  REFERENCE 
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QUALITY  ASSURANCE 


SEC 


SEC  # 


PAGE 


PURPOSE 


Y/N 


PPS 

4. 

2-100 

QA  PROVISIONS 

N 

PDD 

4. 

2-149 

QA  PROVISIONS 

N 

TPL 

9. 

3-14 

EVALUATE  TEST  RESULTS 

N 

TS 

9. 

3-26 

SPECIFY  TEST  REPORTS 

N 

REMARKS 

VALIDATION  NEE 
VALIDATION  NEE 

LOOSE 
VALIDATION  NEE 


SUBPROGRAMS  &  SUBROUTINES 


PDS 

3.4.3 

2-131 

REFERENCE  CONTROL 

Y 

PDS 

6. 

2-133 

COMMON  SUBROUTINES 

Y 

PDD 

1.0 

2-137 

SUBPROGRAM  STRUCTURE 

Y 

PDD 

3.5 

2-148 

LIBRARY  SUBROUTINES 

Y 

PDD 

3.6 

2-148 

INITIATION 
PROGRAMMING 

Y 

SOD 

PDS 
PDS 


6.1 
3.3 
3.5 


2-29       STANDARDS 

2-127/8    MEM.  &  PROC.  ALLOC, 

2-131      LANGUAGE 


Y 
Y 
Y 


FOS 
PPS 


5.2.2 
3.5 


CAPACITY  REQUIREMENTS 

2-39       CONSOLE  CAPACITY 
2-100      PROGRAM  CAPACITY 


Y 
Y 


SOS 
PDS 
OM 
PDM 


4.2 
3.5 

3.1 
3 


CONFIGURATION  MANAGEMENT 

2-14       PHYSICAL  CONFIGURATION  Y 

2-132      PROGRAM  CONFIGURATION  Y 

4-8        MINIMUM  CONFIGURATION  Y 

4-17       SUBPROGRAM  CONFIGURATION  Y 


OM 

4.3 

OM 

6 

OM 

7 

OPERATION 

4-9        PARAMETER  VALUES 
4-9        TROUBLE  REPORTS 
4-10       RECOVERY 
18 


INPUTS 
Section  3.3.1  (p.  2-37)  of  FOS  reads:   "Data  type  by  source,  its  period- 
icity of  update  rate,  and  expected  and/or  reliability  will  be  provided 
in  this  paragraph."   The  meaning  of  the  underlined  words  is  unclear. 

FUNCTIONS 
Sections  3.3  (p.  2-90),  3.3.5  (p.  2-92)  and  3.4  (p.  2-95)  of  PPS  overlap 
to  some  extent. 

DATABASE 
Section  3.5  (p.  2-99)  of  PPS  reads:   "Adaptation  data  is  that  data  that 
can  be  centrally  modified  as  needed  to  define  the  scope  of  operational 
functions  within  prescribed  limits."   The  meaning  of  this  statement  is 
unclear . 

Section  3.3.2  (p.  2-145) ,  Variables  of  PDD  refers  to  "program"  in  line  2 
and  "constant"  under  "a.  constant  name."   The  word  "variable"  was 
probably  intended  in  both  cases. 

INTERFACES 
Sections  5.2  (p.  2-14)  and  5.3  (p.  2-15)  of  SOS  provide   insufficient 
information  concerning  what  comprises  a  peripheral  systems  interface 
and  operator  interface,  respectively. 

Section  5.3  (p.  2-28)  of  SOD  references  a  Peripheral  System  Interface 

section  5.2  rather  than  spelling  out  what  is  requred  for  the  Operator 

Interface. 
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INTERFACES  (CONTINUED) 
Section  1.0  (p. 2-41)  of  IDS  reads:   "This  specification  establishes  a  set 
of  requirements  for  the  preparation  of  a  document  for  defining  the  design 
interdigital  processor  digital  interfaces . "   The  underlined  portion  is 
confusing. 

TESTING 
Section  1.0  (p.  3-5)  of  TPL  reads:   "The  test  plan  shall  define  the  scope 
of  tests  required  to  ensure  that  the  system,  function,  and/or  program 
meets  all  applicable  technical ,  operational  and  performance  specification." 
Since  the  only  categories  of  speciation  in  3560.1  are  "operational," 
"performance,"  and  "design,"  the  meaning  of  "technical"  is  not  clear. 

Section  1.0c  (p.  3-6)  of  TPL  (Function  Test)  and  Section  l.Od  (p.  3-6) 
should  stipulate  validation  against  the  TOR  in  addition  to  verification 
against  performance  and  design  specifications  and  program  description 
document,  respectively. 

Section  1.0  (p.  3-35)  of  TPR  reads:   "Test  procedures  provide  for  the 
quantitative  results  of  tests ,  which  are  later  extracted  for  the  tests 
themselves."   The  meaning  of  this  statement  is  not  clear. 

Sections  3.2. 3 j  (p.  3-42)  of  TPR  reads:   "The  procedure  must  agree 
exactly  with  the  program  behavior."   The  meaning  of  this  statement  is 
not  clear. 
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TESTING  (CONTINUED) 
Sections  1.1.1  (p.  3-45)  and  1.  (p.  3-49)  of  TR  imply  the  allowance  of 
patches  in  programs  during  testing.   This  is  believed  to  be  a  poor 
practice. 

OUALITY   ASSURANCE 
Section  4.  (p.  2-100)  of  PPS ,  Section  4.  (p.  2-149)  of  PDD,  and  Section 
9.  (p.  3-26)  of  TS  should  stipulate  validation  against  the  TOR  in  addition 
to  program  verification. 

Section  9.  (p.  3-14)  of  TPL  should  contain  a  statement  that  government 
personnel  must  witness  the  tests.   Although  this  section,  as  written,  is 
concerned  with  procurement  rather  than  maintenance ,  the  argument  for 
using  government  witnesses  is  valid  for  maintenance. 
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V.   CONCLUSIONS  AND  RECOMMENDATIONS 

Based  on  the  foregoing  analysis,  the  following  conclusions  and 
recommendations  are  presented  relative  to  the  usability  of  3560.1  for 
software  maintenance . 

1.  The  standard  is  comprehensive  and  detailed.   Considering  the  fact 
that  it  was  issued  in  1974,  it  was  unable  to  benefit  from  the  hindsight 
that  is  expressed  in  this  report,  which  is  based  mainly  on  advances  that 
have  been  made  in  programming  methodology  since  1974.   If  3560.1  were 
updated  to  reflect  new  programming  technology,  it  could  be  a  more  com- 
prehensive standard  than  MIL-STD  1679 .   Notable  aspects  of  the  standard 
are  the  following: 

Applicable  Documents  statements. 

Resource  budgets  (time,  space) . 

Numerous  examples . 

Content  Check  Lists. 

Interfaces  descriptions. 

Test  coverage . 

Detailed  Table  of  Contents  for  each  specification. 

2.  As  pointed  out  in  Section  III,  there  are  some  deficiences  in  the 
vital  area  of  traceability . 

3.  As  demonstrated  in  Section  IV,  there  should  be  more  emphasis  on 
validation.   To  quote  from  page  14  of  3560.1:   "The  Tactical  Operational 
Requirement  shall  serve  as  a  life  cycle  configuration  management  device 
for  specifying  the  overall  tactical  operational  software  capability 
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requirements  for  the  combat  system."   That  being  the  case,  there  should 

be  more  emphasis  on  validating  against  the  TOR,  with  respect  to  QA 

procedures,  during  both  development  and  maintenance. 

4.   There  are  many  examples  of  redundancies  and  use  of  inconsistent 

terminology  in  the  standard.   Examples  of  the  former  are: 

Section  7.   Data  Unit  Descriptions,  p.  2-57  and  Section  8. 
Message  Descriptions,  p.  2-69  of  Interface  Design  Specification 
contain  similar  material. 

Test  specification  System,  pp.  3-20  to  3-26  and  Test  Specifica- 
tion Function,  pp.  3-27  to  3-31  contain  similar  material. 
Much  of  the  material  in  the  Test  Plans  and  Test  Specifications 
is  similar.   These  could  be  combined  into  a  single  document, 
with  resultant  reduction  in  verbage  and  increase  in  clarity. 
Each  of  the  documents  is  described  in  the  format  of  purpose, 
scope,  typical  content,  etc.  followed  by  the  actual  format  of 
document  content.   Much  of  the  material  is  duplicated  between 
the  two  sections  (e.g.,  Test  Procedures  pp.  3-35  to  3-37  and 
pp.  3-39  to  3-42) .   This  material  could  be  combined  in  many 
of  the  documents . 

Examples  of  inconsistent  terminology  are: 

Section  1.0  Purpose,  p.  2-153  and  Section  9.  Notes,  p.  2-162 

of  Data  Base  Design  refer  to  a  Subprogram  Description  Document. 

This  document  is  not  defined  or  described  in  3560.1  by  that 

name. 

-   Section  1.1.3  Analysis,  p.  3-46  of  Test  Reports  refers  to 

Operational/Functional  Specification.   This  specification  is 

not  defined  or  described  in  3560.1  by  that  name. 
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5 .  The  standard  is  much  more  oriented  to  software  development  than 
to  software  maintenance.   It  also  seems  to  have  a  strong  orientation  to 
the  Navy  Tactical  Data  System.   A  more  general  orientation  might  be 
preferable  in  order  to  achieve  wider  applicability  to  a  variety  of 
software  systems . 

6.  It  would  be  useful  for  both  software  development  and  maintenance 
activities  to  provide  a  section  in  the  standard  that  describes  the 
material  in  subject  matter  instead  of  document  format,  i.e.,  a  descrip- 
tion of  all  document  sections  that  refer  to  inputs ,  all  that  refer  to 
outputs,  etc.   Such  a  breakdown  was  used  in  TABLE  2  of  this  report. 

7.  In  summary,  an  inexperienced  programmer  would  probably  have 
difficulty  applying  3560.1  to  maintenance  because  of  the  problems 
mentioned  in  this  report.   Because  of  the  great  shortage  of  skilled 
software  personnel,  the  criterion  of  usability  of  the  standard  by  an 
inexperienced  programmer  is  appropriate . 
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DISCLAIMER 

The  opinions  expressed  in  this  report  are  strictly  those  of  the 
author  and  do  not  necessarily  reflect  the  opinions  of  the  Naval 
Postgraduate  School,  Department  of  the  Navy,  or  Department  of  Defense 


25 


DISTRIBUTION    LIST 

Mo.    Copies 

Professor  Lyle  Cox  1 

Code  5  2  CI 

Computer  Science  Department 

Naval  Postgraduage  School 

Monterey,  CA  93940 

Mr.  William  Ferreira  1 

Trident  CCSMA 

Building  12 

U.S.  Navy 

Newport,  R.I.  02840 

Mr.  Gerald  Goulet  1 

Naval  Air  Development  Center 
Warminister,  PA  18974 

Professor  Carl  R.  Jones  1 

Code  54Js 

Administrative  Sciences  Department 

Naval  Postgraduate  School 

Monterey,  CA  93940 

Professor  Norman  Lyons  1 

Code  54Lb 

Administrative  Sciences  Department 

Naval  Postgraduate  School 

Monterey,  CA  93940 

LCDR  Ronald  W.  Modes  1 

Code  52Mf 

Computer  Science  Department 

Naval  Postgraduate  School 

Monterey,  CA  93940 

Mr.  Steven  Oxman  1 

Trident  CCSMA 

Building  132T 

U.S.  Navy 

Newport,  R.I.  02840 

Mr.  Richard  Pariseau  1 

Naval  Air  Development  Center 
Warminster,  PA  18974 


26 


No.  Copies 

Professor  Norman  F.  Schneidewind  10 

Code  54Ss 

Administrative  Sciences  Department  and 

Computer  Science  Department 

Naval  Postgraduate  School 

Monterey,  CA  93940 

Dr.  John  Stancil  1 

Trident  CCSMA 

Building  132T 

U.S.  Navy 

Newport,  R.I.  02840 

Professor  Roger  Weissinger-Baylon  1 

Code  54Wr 

Administrative  Sciences  Department 

Naval  Postgraduate  School 

Monterey,  CA  93940 

AS /OR  Library  1 

Code  54/55 

Naval  Postgraduate  School 

Monterey,  CA  93940 

Computer  Center  Library  2 

Code  0141 

Naval  Postgraduate  School 

Monterey,  CA  93940 

Defense  Technical  Information  Center  2 

Cameron  Station 
Alexandria,  VA  23314 

Knox  Library  4 

Code  0142 

Naval  Postgraduate  School 

Monterey,  CA  93940 

Office  of  Research  Administration  1 

Code  012A 

Naval  Postgraduate  School 

Monterey,  CA  93940 

Computer  Science  Department  2 

Code  52 

Naval  Postgraduate  School 

Monterey,  CA  93940 


27 


DUDLEY  KNOX  LIBRARY 


3  2768  00336488  6 


